Instructor : Bogdan Chlebus Lecture 20 : Markov Chains

If X0 = i then we say that i is the initial state of process X. If Xn = j then process X is in state j in step n, for n > 0. If Xn−1 = k and Xn = ` then process X enters state ` from state k in step n, for n > 0. Let us consider two examples of a process X = (X0, X1, X2, . . .) where Xi ∈ {0, 1}. One is when the random variables X0, X1, X2, . . . are independent and determined by tosses of a coin. Another is when the random variables X0, X1, X2, . . . are determined by time completely, for instance, X2i = 0 and X2i+1 = 1, for all natural numbers i.